Predictive encoding of video data is based on the division of frames into blocks of pixels. For each block of pixels, a predictor block is searched for in available data. The predictor block may be a block in a previous frame, or generated from neighboring, previously encoded/decoded pixels in the current frame. Different encoding modes are defined according to different ways of determining the predictor block. The result of the encoding is an indication of the predictor block and a residual block consisting in the difference between the block to be encoded and the predictor block.
The encoding of residual blocks is based on three main steps consisting in transformation, quantization and finally entropy encoding of the remaining information.
Transformation consists in defining a new partition of the frame into blocks that are here called “transform unit”, according to the HEVC vocabulary, and then applying a transform to the transform unit elements. The transformation is typically aimed at transforming the spatial values in the transform unit to frequency coefficients resulting in a transform unit comprising coefficients.
Quantization consists in scaling the magnitudes of these coefficients, ideally setting a majority of them to zero when the resulting degradation of the image is considered acceptable. High frequency coefficients are often good candidates to be set to zero based on the fact that they define fine details in the image that can be discarded without degrading the resulting image too severely.
Quantization of a transform coefficient typically consists in determining a scaled coefficient based on the transform coefficient by multiplying the transform coefficient by a factor parameter and next adding a deadzone parameter. The scaled coefficient is then shifted to obtain the quantized coefficient. The deadzone parameter allows controlling the rounding of the quantized coefficient and thus impacting the encoding result. The thorough determination of the deadzone parameter is therefore an issue of encoding processes.